The Linear Programming Set Covering Machine

The Set Covering Machine (SCM) was introduced by Marchand & Shawe–Taylor [6, 7] in which a minimum set cover of a class of examples was approximated to find a compact conjunction/disjunction of features for classification. Their approach was to solve the set cover problem using the greedy algorithm. In this paper we introduce an alternative method of solving the SCM by formulating it as a Linear Programme (LP). In this setting we can apply an LP solver to give us our set of data-dependent features and use a convex combination of these features in order to classify unseen data for both the conjunction and disjunction case. Our hope is to approximate better solutions to the set cover problem using an LP as opposed to the greedy method approach evaluated in [6, 7]. The LP formulation is motivated by the LPBoost algorithm and so we also apply boosting algorithms, LPBoost and AdaBoost, to our set of features in order to compare our results with the original SCM and Support Vector Machine(SVM) classifiers.